Multi-sensor Data Fusion Techniques Research Articles

SENTINEL GUARD: A FATIGUE DETECTION SYSTEM WITH MULTI-CHANNEL ALERT INTEGRATION

The fatigue detection system is a critical application aimed at enhancing safety in various domains by addressing the risks associated with drowsiness related incidents. Leveraging advanced technologies such as machine learning, computer vision, and real-time communication, the system offers a comprehensive solution for monitoring and alerting users to signs of fatigue in real-time. Through the integration of pre-trained models, such as the shape_predictor_68_face_landmarks.dat, the system accurately detects facial landmarks and tracks eye movements, enabling the identification of drowsiness indicators. Upon detection, the system generates alerts via email and WhatsApp, accompanied by audible alarm buzzers to prompt immediate user intervention. The inclusion of live location information in alerts provides additional context, facilitating timely assistance and support. Furthermore, the system's architecture allows for future enhancements, including the integration of multi-sensor data fusion techniques and collaboration with industry stakeholders to promote standardized protocols and widespread adoption. Overall, the fatigue detection system represents a significant step towards mitigating the risks of drowsinessrelated accidents and improving safety across various applications Key Words: Fatigue detection, Machine learning, CNN, Real time alerts, Facial Expression analysis, Open CV

Condition monitoring of Industrial machine’s transmission elements – MSDF feature extraction approach

ABSTRACT Predictive maintenance is a modern Industry 4.0 strategy in which machines are continuously monitored to detect flaws and prevent breakdowns before they occur. A single sensor focuses on a single variable while neglecting larger information components, resulting in poor data quality and an increased risk of issues in critical rotating equipment. Multi-sensory configuration technology has been developed to collect huge amounts of data from a machine in order to enhance monitoring capabilities in terms of precision, resolution, efficiency, resilience, and trustworthiness of the overall system. The goal of this work is to provide an integrated perspective on machine monitoring using the Multi Sensor Data Fusion (MSDF) technique. On four fault bearings, a case study contrasts the results of single and multiple sensors. A feature-level data fusion method is used, in which computations using time-domain vibration signature data are utilised to build a fusional vector, which is then classified using SVM and analysed with Gaussian kernels. The experimental results suggest that the proposed Gaussian kernel with SVM technique outperforms single sensor data interpretation in terms of classification accuracy and generalisation capability. It is an efficient way for finding defects in rotating machinery in excessively noisy environments.

The Multisensor Data Fusion Method Based on Improved Fuzzy Evidence Theory in the Coal Mine Environment

An enhanced evidence theory-based multisensor data fusion technique is presented to address the problem of poor data fusion caused by an unknown interference in the fully automated mining face multisensor system of a coal mine. Initially, the set of all measurement values is considered as the identification framework, and the principles of fuzzy mathematics are applied to introduce the membership function. This leads to the proposal of a novel method for calculating mutual support among multiple sensors. Furthermore, the basic belief assignment (BBA) in evidence theory is determined by measuring the confidence distance between sensors. Subsequently, a divergence measure is employed to assess the level of conflict and difference between BBA functions, which serves as an indicator of their credibility. The credibility of BBA functions is further adjusted by calculating their information volume using Shannon entropy. This adjustment aims to increase the weight of BBA functions that exhibit less conflict with other BBA functions. Ultimately, the fusion result is obtained through an evidence combination rule based on a conflict allocation. The numerical experimental results demonstrate that the proposed approach achieves higher accuracy, better robustness, and generality compared to the existing methods.

Real-time deformation monitoring of large diameter shield tunnel based on multi-sensor data fusion technique

The measurement of the shield tunnel deformation is an important tool to ensure structural integrity. The structural “deformation” of the tunnel is not merely a simple geometric deformation, which includes the elastic deformation of the segments and the rigid deformation caused by the opening and misalignment of the joints between the segments. Therefore, it is hard to monitor the deformation only relying on a single device. In this study, a data-fusion computing algorithm based on a fusion of rotation and strain measurements is proposed to improve the accuracy of tunnel deformation monitoring without the need of reference frame. SmartRock sensor was utilized to record rotation and the strain change of tunnel segments. Kalman filter was used as the fusion technique. The algorithm was successfully applied in the project of Jinan Yellow River tunnel, China. The result was compared with the result obtained by laser scanner and shows good consistency.

Multi-Sensor Data Fusion Solutions for Blind and Visually Impaired: Research and Commercial Navigation Applications for Indoor and Outdoor Spaces

Several assistive technology solutions, targeting the group of Blind and Visually Impaired (BVI), have been proposed in the literature utilizing multi-sensor data fusion techniques. Furthermore, several commercial systems are currently being used in real-life scenarios by BVI individuals. However, given the rate by which new publications are made, the available review studies become quickly outdated. Moreover, there is no comparative study regarding the multi-sensor data fusion techniques between those found in the research literature and those being used in the commercial applications that many BVI individuals trust to complete their everyday activities. The objective of this study is to classify the available multi-sensor data fusion solutions found in the research literature and the commercial applications, conduct a comparative study between the most popular commercial applications (Blindsquare, Lazarillo, Ariadne GPS, Nav by ViaOpta, Seeing Assistant Move) regarding the supported features as well as compare the two most popular ones (Blindsquare and Lazarillo) with the BlindRouteVision application, developed by the authors, from the standpoint of Usability and User Experience (UX) through field testing. The literature review of sensor-fusion solutions highlights the trends of utilizing computer vision and deep learning techniques, the comparison of the commercial applications reveals their features, strengths, and weaknesses while Usability and UX demonstrate that BVI individuals are willing to sacrifice a wealth of features for more reliable navigation.

Robust Unscented Kalman Filter-Based Decentralized Multisensor Information Fusion for INS/GNSS/CNS Integration in Hypersonic Vehicle Navigation

INS/GNSS/CNS (inertial navigation system/global navigation satellite system/celestial navigation system) integration is a favourable navigation mode for hypersonic vehicles. However, since the measurements from GNSS and CNS are easily interfered during highly dynamic maneuvers, this integration is very difficult to achieve optimal navigation solution with existing information fusion techniques. This paper proposes a new method of decentralized multi-sensor information fusion based on robust UKF (unscented Kalman filter) for INS/GNSS/CNS integration to solve the above issue. Firstly, a fault detection-based robust UKF is established for local state estimation, in which a hypothesis test is constructed via the Mahalanobis distance of innovation to detect abnormal measurements in GNSS and CNS; and subsequently, a scalar factor is determined and further introduced into the innovation covariance to decrease the Kalman gain to improve the UKF robustness against abnormal measurements. Secondly, the traditional multi-sensor optimal data fusion technique is extended to nonlinear systems by use of unscented transformation in the framework of minimum variance estimation to fuse the local state estimations from INS/GNSS and INS/CNS subsystems. The proposed information fusion method can achieve the globally optimal fusion estimation results against abnormal measurements for hypersonic vehicle navigation with INS/GNSS/CNS integration. Semi-physical simulations and comparison analysis have validated the superior performance of the proposed method.

Multisensor Information Fusion for Condition Based Environment Monitoring

Destructive wildfires are becoming an annual event, similar to climate change, resulting in catastrophes that wreak havoc on both humans and the environment. The result, however, is disastrous, causing irreversible damage to the ecosystem. The location of the incident and the hotspot can sometimes have an impact on early fire detection systems. With the advancement of intelligent sensor-based control technologies, the multi-sensor data fusion technique integrates data from multiple sensor nodes. The primary objective to avoid wildfire is to identify the exact location of wildfire occurrence, allowing fire units to respond as soon as possible. Thus to predict the occurrence of fire in forests, a fast and effective intelligent control system is proposed. The proposed algorithm with decision tree classification determines whether fire detection parameters are in the acceptable range and further utilizes a fuzzy-based optimization to optimize the complex environment. The experimental results of the proposed model have a detection rate of 98.3. Thus, providing real-time monitoring of certain environmental variables for continuous situational awareness and instant responsiveness.

Hurricane and Typhoon Storm Wind Resolving NOAA NCEI Blended Sea Surface Wind (NBS) Product

Improving forecasts of storms and hurricanes and their potential impacts is highly important to public safety, economic security, commerce, and community infrastructure. One key element of forecast improvement is more accurate and increased spatial–time coverage of observational data for model calibration, quality control and initialization, and/or data assimilation. The National Oceanic and Atmospheric Administration (NOAA) has been producing a global gridded 0.25° and 6-hourly sea surface winds product that has wide applications in marine transportation, marine ecosystem and fisheries, offshore winds, weather and ocean forecasts, and other areas. The NOAA National Centers for Environmental Information (NCEI) Blended Sea winds (NBS) v1.0 product is generated by blending observations from multiple sources (satellites), including scatterometers and microwave radiometers/imagers. However, these sensors do not provide accurate observations of intensive high-speed hurricane winds because their signals saturate in very high winds or degrade in the presence of rain. Recent advancements in satellite wind retrievals revealed that the L-band (1.42 GHz) instrument on the Soil Moisture Active Passive (SMAP) satellite and the AMSR2 All-Weather channel (~6.9 GHz) can provide accurate hurricane winds of up to 65 m/s (145 MPH) without being affected by rain; these data are incorporated in a new version of the Blended Sea Winds, NBS v2.0, using a multi-sensor data fusion technique based on random errors, enabling it to resolve very high winds, especially along the eyewalls of tropical cyclones and hurricanes. NBS v2.0 provides both a long-term record of 30+ years retrospectively since July 1987 and a near-real-time mode with 1-day latency.

DEVELOPMENT OF AUTOMATED SATELLITE DATA DOWNLOADING AND PROCESSING PIPELINE ON AWS CLOUD FOR NEAR-REAL-TIME AGRICULTURE APPLICATIONS

Abstract. Remote sensing satellites allow users to acquire detailed information about the Earth's surface on a temporal basis. Widen time-series analysis at a large geographical scale involves a huge amount (in Terabytes) of satellite data downloading and processing operations. Such processes need good computational power, large storage, and sophisticated tools. Maintaining such infrastructure can cost heavily to the research/commercial enterprises. To overcome such issues, Amazon Web Service (AWS) offers a sophisticated cloud computing environment. We developed an in-house automated satellite data downloading and processing (ADDPro) pipeline on the AWS platform. The ADDPro pipeline employed Sentinel-2 satellite data to offer current and relative vegetation health information of the agriculture region on a temporal basis at the pan-India scale. Image compositing and multi-sensor data fusion technique have been incorporated into the ADDPro pipeline to produce cloud-free raster (GeoTIFF) outputs. ADDPro pipeline also facilitates lossless raster data compression, which reduces AWS data transfer costs between regions. Data compression also aids in reducing raster publishing time on GeoServer. Operationally, AWS allows users to download only the bands required to generate a certain index (e.g. NDVI) rather than the entire Sentinel-2 data package. The entire ADDPro pipeline is extremely cost-effective, efficient, and scalable.

A Detachable and Expansible Multisensor Data Fusion Model for Perception in Level 3 Autonomous Driving System

Various sensors are adopted by autonomous driving systems to perceive objects and surroundings. Thus, the multi-sensor data fusion techniques become essential to combine different sensors’ advantages for better perception performance. However, the current multi-sensor data fusion techniques suffer from the high cost of computation resources, low expansibility for more diverse sensors, and insufficient systematic consideration for modeling. This paper first constructs a detachable and expansible multi-sensor data fusion model based on three main modules: front fusion, global fusion, and synthesizer, where the methods for flexible association gating and virtual targets have been designed. The model can be disassembled and configured for different trim levels of vehicles and is easily expansible for adding more heterogeneous sensors. Next, the presented multi-sensor data fusion model is compared with the cheap Joint Probabilistic Data Association (C-JPDA) method. The comparison shows the superior accuracy of the designed model on false association and effective narrowing of the variance of object detection. Finally, the presented multi-sensor data fusion model is integrated into an embedded system and experimented on urban roads and highways with the engaged Level 3 autonomous driving function. The experiment results indicate that the proposed model has excellent sensor data fusion performance and provides accurate and timely object information in the Level 3 autonomous driving system.

Performance prediction of cryogenic/semi-cryogenic rocket engines by using multi-sensor data fusion techniques

Performance prediction of cryogenic/semi-cryogenic rocket engines by using multi-sensor data fusion techniques

Smart-City-based Data Fusion Algorithm for Internet of Things

Increasingly, Wireless Sensor Networks (WSNs) are contributing enormous amounts of data. Since the recent deployments of wireless sensor networks in Smart City infrastructures, significant volumes of data have been produced every day in several domains ranging from the environment to the healthcare system to transportation. Using wireless sensor nodes, a Smart City environment may now be shown for the benefit of residents. The Smart City delivers intelligent infrastructure and a stimulating environment to citizens of the Smart Society, including the elderly and others. Weak, Quality of Service (QoS) and poor data performance are common problems in WSNs, caused by the data fusion method, where a small amount of bad data can significantly impact the total fusion outcome. In our proposed research, a WSN multi-sensor data fusion technique employing fuzzy logic for event detection. Using the new proposed Algorithm, sensor nodes will collect less repeated data, and redundant data will be used to increase the data's overall reliability. The network's fusion delay problem is investigated, and a minimum fusion delay approach is provided based on the nodes’ fusion waiting time. The proposed algorithm performs well in fusion, according to the results of the experiment. As a result of these discoveries, It is concluded that the algorithm describe here is effective and dependable instrument with a wide range of applications.

Urban tree species classification using UAV-based multi-sensor data fusion and machine learning

ABSTRACT Urban tree species classification is a challenging task due to spectral and spatial diversity within an urban environment. Unmanned aerial vehicle (UAV) platforms and small-sensor technology are rapidly evolving, presenting the opportunity for a comprehensive multi-sensor remote sensing approach for urban tree classification. The objectives of this paper were to develop a multi-sensor data fusion technique for urban tree species classification with limited training samples. To that end, UAV-based multispectral, hyperspectral, LiDAR, and thermal infrared imagery was collected over an urban study area to test the classification of 96 individual trees from seven species using a data fusion approach. Two supervised machine learning classifiers, Random Forest (RF) and Support Vector Machine (SVM), were investigated for their capacity to incorporate highly dimensional and diverse datasets from multiple sensors. When using hyperspectral-derived spectral features with RF, the fusion of all features extracted from all sensor types (spectral, LiDAR, thermal) achieved the highest overall classification accuracy (OA) of 83.3% and kappa of 0.80. Despite multispectral reflectance bands alone producing significantly lower OA of 55.2% compared to 70.2% with minimum noise fraction (MNF) transformed hyperspectral reflectance bands, the full dataset combination (spectral, LiDAR, thermal) with multispectral-derived spectral features achieved an OA of 81.3% and kappa of 0.77 using RF. Comparison of the features extracted from individual sensors for each species highlight the ability for each sensor to identify distinguishable characteristics between species to aid classification. The results demonstrate the potential for a high-resolution multi-sensor data fusion approach for classifying individual trees by species in a complex urban environment under limited sampling requirements.

Comparative analysis of volatile organic compounds of breath and urine for distinguishing patients with liver cirrhosis from healthy controls by using electronic nose and voltammetric electronic tongue

Advanced stage detection of liver cirrhosis (LCi) would lead to high mortality rates in patients. Therefore, accurate and non-invasive tools for its early detection are highly needed using human emanations that may reflect this disease. Human breath, along with urine and blood, has long been one of the three main biological media for assessing human health and environmental exposure. The primary objective of this study was to explore the potential of using volatile organic compounds (VOCs) assay of exhaled breath and urine samples for the diagnosis of patients with LCi and healthy controls (HC). For this purpose, we used a hybrid electronic nose (E-nose) combining two sensor families, consisting of an array of five commercial chemical gas sensors and six interdigitated chemical gas sensors based on pristine or metal-doped WO3 nanowires for sensing volatile gases in exhaled breath. A voltammetric electronic tongue (VE-tongue), composed of five working electrodes, was dedicated to the analysis of urinary VOCs using cyclic voltammetry as a measurement technique. 54 patients were recruited for this study, comprising 22 patients with LCi, and 32 HC. The two-sensing systems coupled with pattern recognition methods, namely Principal Component Analysis (PCA) and Discriminant Function Analysis (DFA), were trained to classify data clusters associated with the health status of the two groups. The diagnostic performances of the E-nose and VE-tongue systems were studied by using the receiver operating characteristic (ROC) method. The use of the E-nose or the VE-tongue separately, trained with these appropriate classifiers, showed a slight overlap indicating no clear discrimination between LCi patients and HC. To improve the performance of both electronic sensing devices, an emerging strategy, namely a multi-sensor data fusion technique, was proposed as a second aim to overcome this shortcoming. The data fusion approach of the two systems, at a medium level of abstraction, has demonstrated the ability to assess human health and disease status using non-invasive screening tools based on exhaled breath and urinary VOC analysis. This suggests that exhaled breath as well as urinary VOCs are specific to a disease state and could potentially be used as diagnostic methods.

Prediction of environment variables in precision agriculture using a sparse model as data fusion strategy

Precision agriculture seeks to optimize production processes by monitoring and analyzing environmental variables. For example, establishing farming actions on the crop requires analyzing variables such as temperature, ambient humidity, soil moisture, solar irradiance, and Rainfall. Although these signals might contain valuable information, it is vital to mix up the monitored signals and analyze them as a whole to provide more accurate information than analyzing the signals separately. Unfortunately, monitoring all these variables results in high costs. Hence it is necessary to establish an appropriate method that allows the infer variables behavior without the direct measurement of all of them.This paper introduces a multi-sensor data fusion technique, based on a sparse representation, to find the most straightforward and complete linear equation to predict and understand a particular variable behavior based on other monitored environmental variables measurements. Moreover, this approach aims to provide an interpretable model that allows understanding how these variables are combined to achieve such results. The fusion strategy explained in this manuscript follows a four-step process that includes 1. data cleaning, 2. redundant variable detection, 3. dictionary generation, and 4. sparse regression. The algorithm requires a target variable and two highly correlated signals. It is essential to point out that the developed method has no restrictions to specific variables. Consequently, it is possible to replicate this method for the semiautomatic prediction of multiple critical environmental variables.As a case study, this work used the SML2010 data set of the UCI machine learning repository to predicted the humidity's derivative trend function with an error rate lower than 17% and a mean absolute error lower than 6%. The experiment results show that even though sparse model predictions might not be the most accurate compared to those of linear regression (LR), support vector machine (SVM), and extreme learning machine (ELM) since it is not a black-box model, it guarantees greater interpretability of the problem.

Multisensor data fusion technique for energy conservation in the wireless sensor network application “condition-based environment monitoring”

Due to the nature of the continuous and large volume of data transmission, energy conservation appears particularly tedious at WSN. This exhausts the energy faster, leading to failure of the sensor network. With the increase in the number of applications, there is high complexity in data transmission and more extensive accuracy requirements over the last few years, extended use of sensors challenges on the increase in sensors battery life-time too. In leveraging the display and control of the sensor operation, the multisensor data fusion technique plays a vital role. In the Condition-based Environment Monitoring System application, the proposed ADKF-DT-MF for the multisensor data fusion is implemented to detect natural and human disturbances to provide accurate and rapid environmental awareness. This paper describes the energy conservation module of the proposed system in concern to accuracy, processing efficiency, energy consumption, and the overall network operational life-time. The simulation results show a better accuracy of 97% with an energy consumption of 0.95 compared with the existing FIM, VWFFA, and Fuzzy algorithms.

A Deep Learning Based Data Fusion Method for Degradation Modeling and Prognostics

Degradation modeling is a critical and challenging problem as it serves as the basis for system prognostics and evolution mechanism analysis. In practice, multiple sensors are used to monitor the status of a system. Thus, multisensor data fusion techniques have been proposed to capture comprehensive information for prognostic modeling and analysis, which aims at developing a composite health index (HI) through the fusion of multiple sensor signals. In the literature, most existing methods use a linear data-fusion model for integration of multisensor data to construct the HI, which is insufficient to model nonlinear relations between sensing signals and HI in a complicated system. This article proposes a novel data fusion method based on deep learning for HI construction for prognostic analysis. A pair of adversarial networks is proposed to enable the training procedure of neural networks. To guarantee the stability of the algorithm, we propose a root mean square propagation (i.e., RMSprop)-based sampling algorithm to estimate model parameters. A set of simulation studies and a case study on a set of degradation signals of aircraft engines are conducted. The results demonstrate that the proposed method has a significant improvement on remaining useful life prediction compared to existing data fusion methods.

Embedded System for Precision Positioning, Detection, and Avoidance (PODA) for Small UAS

This article described the implementation and development of positioning, detection, and avoid (PODA), an embedded electronic platform, installed on-board a UAV, designed for precision positioning. The experimental campaigns showed that the data fusion LIDAR/ADXL345 could assist a UAV during navigation, as well as in critical conditions, such as low visibility (i.e., fog, high concentration of smoke in case of fire). Further work will be related to PODA performance enhancement, in particular: Accurate accelerometer calibration with a least square method; Obstacle detection; - Additional sensors installed, such as inertial measurement units and time-of-flight sensors; Multisensor data fusion techniques and fault detection analysis; and Experimental sessions with faster data acquisition rates (i.e., sampling at 50 Hz) and simulation of different landing conditions (with and without obstacles in landing area).

A review on multi sensor data fusion technique in CNC machining of tailor-made nanocomposites

In this study, a multiple sensor data fusion system is anticipated as essential for monitoring of cutting operations, by identifying suitable sensor locations to obtain feedback signals periodically. The sensor signal derives the failure during machining owing to complex cutting tool geometry even in machining of composites. Nano metal matrix composites (NMMC) being extremely upright in mechanical characteristics, consequently machining of these hybrid nano metal matrix composites reinforced with difficult-to-cut nano particles leads to reduced tool life thereby causing rapid flank wear. Therefore, it is a challenge to identify the wear features caused during machining of tailor made NMMC’s, reducing wastage and preventing machine malfunction. This paper presents a comprehensive review on the machining strategies in extreme output conditions which rely on input parameters of speed, feed and depth of cut influencing tool life during CNC machining. This can be achieved only with multiple sensor data fusion technique during CNC machining.

Cognitive IoT system with intelligence techniques in sustainable computing environment

Forest border crossing animals creates major societal related issues, in addition to endangering their own lives. This is the objective focused in this paper targeting the species “The Elephant”, incorporating with technical methodologies namely, multi-sensor data fusion, cognition theories and computational intelligence techniques. Multi-sensor data fusion provides three level detection of target, along with its related outputs, which improves performance metrics. Cognition theory resulted in obtaining other interesting features about the target. Computational intelligence techniques integrate and conclude the presence of the target in the pseudo-boundary. The technical combination enhances the novelty of the research work, resulting in achieving remarkable accuracy and minimized false alert. An IoT kit was designed and deployed in the real time wild environment in Hosur forest region for collecting the data of Elephant. Further, the notification is sent to the registered mobile of the forest authority, as an early warning for chasing the pachyderm back to the forest.